Papers with representation model
LITE: Intent-based Task Representation Learning Using Weak Supervision (2022.naacl-main)
Copied to clipboard
| Challenge: | To-do texts are often short and under-specified, which poses a challenge for current text representation models. |
| Approach: | They propose a neural multi-task learning framework that extracts representations of English to-do tasks with a multi-head attention mechanism on top of a pre-trained text encoder. |
| Outcome: | The proposed model outperforms baseline models on four downstream tasks and achieves error reduction of 38.7%. |
Learning Disentangled Textual Representations via Statistical Measures of Similarity (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to disentangle a sensitive attribute from textual representations require training and multiple parameter updates. |
| Approach: | They propose a family of regularizers for learning disentangled representations that do not require training. |
| Outcome: | The proposed regularizers are faster and faster and achieve better results when combined with pretrained and randomly initialized text encoders. |
Fine-Tuning a Pre-Trained Wav2Vec2 Model for Automatic Speech Recognition- Experiments with De Zahrar Sproche (2024.lrec-main)
Copied to clipboard
| Challenge: | Developing semi-automatic methods of transcription and annotation based on small amounts of annotated data would free field linguists to focus on tasks that are linguistically and relationally significant during fieldwork. |
| Approach: | They propose to use a pre-trained model to tune a generic pre-trainer model to reduce the transcription workload of field linguists. |
| Outcome: | The proposed system reduces the transcription workload of field linguists by averaging a pre-trained model with a language-specific tuning. |